In recording a sequence of images of a scene, there often occur movements of a camera relative to the scene (for instance translations and/or rotations). Consequently, the place where objects and a background of these objects are imaged in consecutive images will change from image to image.
For various applications, it is necessary to determine these movements. An example of such an application is the elimination of the effect of shaking movements of the camera. This requires determination of the magnitude of the shaking movements. It can be determined, for instance, from the movement of the background of the scene in the image.
U.S. Pat. No. 6,005,639 discloses a technique for determining the movement of the background and objects from one image to the next image in a video signal. To that end, a measure is calculated for the difference between the image content of possibly corresponding areas in successive images, whilst the locations of the areas in the different images differ from each other by a displacement. A displacement leading to a minimum value of the measure for the difference is utilized as displacement vector of the object or the background imaged in the areas.
Movements of objects relative to the background lead to a problem in this technique. This is because as a result of such movements, in each case parts of the background in the images are covered or, conversely, appear. Such movements can be the result of “real” movements of objects and of apparent movements resulting from perspective effects in camera movements.
U.S. Pat. No. 6,005,639 solves this problem by making use of four consecutive images and a series of uniformly displaced areas in those images. That is, the difference in the location of the areas in immediately consecutive images in each case corresponds with the same displacement vector. If the image content of the areas in all these areas differs minimally, the displacement vector is used as displacement vector of the object or the background. A different case is that the image content differs minimally between areas from a few images in the sequence, but differs much more from areas from one or more other images of the sequence. In this case, the image content is apparently covered in the one or more other images, but the displacement vector can nonetheless be used as displacement vector of the object or the background.
Thus, the effect of coverage of the background is controlled. However, this is done under the assumption that the background and the objects move uniformly. This assumption does not always apply to shaking movements. Also, this assumption limits the length of the sequence of images on which the technique of U.S. Pat. No. 6,005,639 is usable. U.S. Pat. No. 6,005,639 utilizes four consecutive images. A much greater number is not usable because of the risk of non-uniform movements, and also because of the large memory occupation needed for this. Consequently, the technique described is not suitable for images in which more prolonged coverage occurs.